21 Using the Testing as a Service Portal

Setting Up Applications

The Testing Cloud Self Service Portal allows Testing Administrators to manage the application infrastructure for testing activities, define user activities for self-service testing, and manage test environments and test assets. Specific responsibilities include:

• Manage the test drivers.

• Manage user roles and access to test drivers (specify which users have access to which plug-in.)

• Manage access to test environments and test assets based on users and roles.

• Assign the OVM and Zone privileges to the following roles and users:

  • OVM - View privilege to Test Designer and Test Administrator

  • Zone - Operator privilege to Test Designer and Test Administrator

  Since TaaS cannot know before hand the zones and Oracle Virtual Machines present in user deployments, the Super Administrator, Cloud Administrator, or respective target owner will decide which zones and Oracle Virtual Machines are to be used.

Note: Ensure the prerequisites are met before using the Testing Cloud Self Service Portal (see Prerequisites).

Follow this process to set up applications to be tested.

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Testing Administrator credentials.

2. From the Setup menu, select Cloud, then select Testing. The Testing Administrator page appears.

3. Click Applications located in the left panel to define applications to be tested.

4. Click Add to define the name and description of the application to make available for testing.

5. Define the name and description and click Save.

Setting Up Oracle Load Testing Test Drivers

Note: Ensure the prerequisites are met before setting up Oracle Load Testing Test Drivers (see Prerequisites).

The Testing as a Service Portal allows you (Testing Administrator) to Register an Oracle Load Testing test driver to be used by Testers during testing. The Test Driver is the package of software that Enterprise Manager uses to generate workload against test system based on the Deployment Plan. To register an Oracle Load Testing Driver, follow these steps:

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Administrator credentials.

2. From the Setup menu, select Cloud, then select Testing. The Testing Administrator page appears.

3. Click Test Drivers located in the left panel. The Test Driver page appears.

4. From the Register menu, select Oracle Load Testing.

5. Enter a meaningful name for this Test Driver.

6. Click the magnifying glass for selecting a deployment plan on which the test driver is based. In the resulting dialog box, select the Deployment Plan that should be used for the Oracle Load Testing test driver and click Select.

   Ensure that Test Designers and Administrators create the deployment plan "on a zone". Also, ensure they have right privileges to this zone. For additional information, see Setting Up Self Update. For information regarding deployment plans, see Oracle Enterprise Manager Lifecycle Management Administrator's Guide.

7. Enter a description for the Test Driver and click Register.

8. Select a Test Driver in the list then click on the component to drill down to the Deployment Plan details.

Setting Up Custom Test Driver Types

Note: Ensure the prerequisites are met before setting up Custom Test Driver Types (see Prerequisites).

The Custom Test Driver allows testers to use test tools from third party vendors together with TaaS. Use Custom Test Driver Types to define and register test-driver types which meet your specific testing needs.

Oracle Load Testing is a type of driver, and since it is created by Oracle, we (Oracle) know the specifics of the driver. However, when you create a test driver, you need to direct your scripts on how to launch and monitor your tests. You can register your own type of test drivers as long as you can build a deployment plan around your assembly which contains the testing product and software.

Note: If you are using third party test automation tools, these tools need to be registered as Custom Test Drivers.

To register a custom test driver, follow these steps:

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Administrator credentials.

2. From the Setup menu, select Cloud, then select Testing. The Testing Cloud Self Service Setup page appears.

3. Click Custom Types located in the left panel. The Custom Test Driver Types page appears

4. Click Create.

5. On the Add Custom Test Driver Type dialog, provide the following information:

   1. Enter a meaningful name and description.

   2. The Working directory is the folder on the virtual machine (VM) created from your test-driver where the operations will occur.

   3. The Results directory is the folder on the VM created from your test-driver and where the results of your tests will appear, TaaS will zip up the entire folder, send it to Enterprise Manager.

   4. Pre-Execution Directive is a file which performs pre-execution steps for your test. Click Browse to locate the file.

   5. Execution Directive is a file which launches the test. Click Browse to locate the file.

   6. Execution Monitoring Directive is a script which TaaS will run on the test driver, to determine if the test is running or complete (and its outcome). Click Browse to locate the file.

      When the test is complete, TaaS reports the outcome of the test. TaaS looks at the value returned by this script to determine the test outcome (passed, failed, warning).

      Note: For TaaS to know the outcome of the trial, ensure that your script prints one of the following outcomes: passed, failed or warning, as the last line to the standard output.

   7. Post-Execution Directive is a file which performs the clean up steps after your test runs. Click Browse to locate the file.

   8. Custom properties file is a property file which you specify values used at run-time.

   Once you provide these inputs, your type is defined and you can now register a new driver of this type.

6. To register a driver, on the Test Drivers page, select Custom from the Register menu.

7. Provide a meaningful name for this Test Driver.

8. Click the magnifying glass for selecting a deployment plan on which the test driver is based. In the resulting dialog box, select the Deployment Plan that should be used for the Custom test driver and click Select.

9. Enter a description for the Test Driver and click Register.

10. Select a Test Driver in the list then click on the component to drill down to the Deployment Plan details.

Setting Up Quotas

Note: Ensure the prerequisites are met before setting up quotas (see Prerequisites).

The Testing as a Service Portal allows you (Testing Administrator) to Setup usage quotas for Testers by specific roles. To set up quotas, follow these steps:

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Administrator credentials.

2. From the Setup menu, select Cloud, then select Testing. The Testing Administrator page appears.

3. Click Quota located in the left panel. The Quotas page appears.

4. Click Create.

5. Click the magnifying glass for selecting roles. In the resulting dialog box, select the role to set quotas for and click Select.

6. Enter the number of Guest Virtual Machines, CPUs, Memory size, Storage size, and number of Virtual Users to allocate to the selected role and click Create.

Publishing Test Environments and Test Assets

Note: Ensure the prerequisites are met before publishing test environments and test assets (see Prerequisites).

The Testing as a Service Portal allows you (Testing Administrator) to Setup usage quotas for Testers by specific roles. To publish test environments and test assets, follow these steps:

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Administrator credentials.

2. From the Setup menu, select Cloud, then select Testing. The Testing Administrator page appears.

3. Click Publish located in the left panel. The Publish page appears.

4. Click Add.

5. From the Add menu in the Select Test Assets, Test Environments, Test Drivers region, select Test Environment. In the resulting dialog box, select the environment to publish and click Select.

6. From the Add menu in the Select Test Assets, Test Environments, Test Drivers region, select Test Asset. In the resulting dialog box, select the test asset to publish and click Select.

7. From the Add menu in the Select Test Assets, Test Environments, Test Drivers region, select Test Driver. In the resulting dialog box, select the test driver to publish and click Select.

8. Click Add in the Select Roles region. In the resulting dialog box, select the role to publish the test asset and environment to and click Select.

9. Click Publish.

Setting Up Chargeback Service

Note: Ensure the prerequisites are met before setting up the chargeback service (see Prerequisites).

Follow this process to set up applications to be tested.

1. Log in to Enterprise Manager Cloud Control using Testing Administrator credentials.

2. From the Setup menu, select Cloud, then select Testing. The Testing Administrator page appears.

3. Click Chargeback located in the left panel to define the chargeback service.

4. Click Configure Charge Plan to define Extended Charge Plans. Plans may be defined by configuration or usage. If defining plans by usage, charges may be based on universal metrics (CPU, memory, storage) or by metrics specific to the targets. Use extended charge plans to set individual prices. For information regarding chargeback, see Oracle Enterprise Manager Cloud Administration Guide.

5. Click Configure Targets to assign Charge Plans. Charge plans must be assigned to targets. If targets are grouped into a zone, then a charge plan can be assigned to the zone, and will apply to all targets within that zone. If there are multiple zones, then different charge plans may be assigned to each one. The Self-Service Application (SSA) administrator can determine chargeback pricing by assigning existing charge plans to the different zones.

6. Click Configure Cost Center to set up Cost Centers (Optional). Cost Centers may be set up to aggregate costs among groups of users, but are not required. If Cost Centers are set up, the chargeback reports by Cost Center are only available to SSA administrators.

Creating Test Assets

A Test Asset is a collection of test artifacts (scenarios, scripts, tests) that can be executed on a test environment. A test asset defines the actions that need to get executed to reproduce the workload against the test environment. The Testing Cloud Self Service Portal allows the Test Designer to create Test Assets based upon the available (Oracle Load Testing- or Custom - Test Drivers) Testing drivers defined by the Testing Administrator.

The Test Designer should know the application under test and its relevant Key Performance Indicators (KPIs). The Test Designer should also know the business use-cases for the test scenarios being run. Specific responsibilities include:

• The Test Designer should be skilled in Oracle OpenScript and Oracle Load Testing if using the Oracle Load Testing Test Driver. If a Custom Test Driver is used, the Test Designer should be skilled using the corresponding test scripting tool.

• The Test Designer should be skilled as DBA.

• Create Test Assets using the Testing as a Service portal.

• Create Test Environments using the Testing as a Service portal.

• If the Test Designer is required to create environments, the Test Designer must know how to create assemblies.

Note: Ensure the prerequisites are met before creating test assets (see Prerequisites).

Follow this process to create test assets.

1. Log in to Enterprise Manager Cloud Control using Test Designer credentials.

2. From the Enterprise menu, select Cloud, then select Self Service Portal. The Self Service page appears.

3. Click My Tests. The My Library tab appears.

4. Click Test Assets to define the test assets to be added to the library.

5. From the Create menu, select the type of asset you want to add, either Oracle Load Testing or Custom. Test Asset Properties page appears.

6. For the Custom asset type, select the type of Custom asset.

   Enter a name and description for the test asset.

7. Click the Application drop-down list and select the application to be tested from the list.

8. Enter any tags to use as keywords to further identify this test asset. These tags make test assets easier to identify and to understand the purpose of the asset.

9. Click Browse and select the asset file you want to upload. The file type for Oracle Load Testing is SCNZIP. The file type for Custom can be whatever file type understood by the test-driver involved.

10. Select or clear the Parameters Exposed for Testing options. These are parameters that are specified as part of the Oracle Load Testing scenario. Parameters selected in this dialog box will be exposed within TaaS and the parameters can be changed for the TaaS test runs.

11. Click OK. The Test Asset is added to the library. Click on a test asset name to drill down to the Test Asset Details page.

Creating Test Environments

A Test Environment is a collection of entities that represents the entire application stack, thus an environment consists of the following elements:

• Application under test (AuT): Contains the software bits and application data. For example, for a deployed Oracle E-Business Suite application, end users can modify settings, configurations, and parameters of the application.

• Infrastructure under test (IaT): These are entities the Application under test depends upon, for example, the cloud setup/infrastructure in a cloud deployment. Typically an infrastructure is something created for consumption of end users by administrators.

Note: Ensure the prerequisites are met before creating test environments (see Prerequisites).

Follow this process to create test environments.

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Test Designer credentials.

2. From the Enterprise menu, select Cloud, then select Self Service Portal. The Self Service page appears.

3. Click My Tests. The My Library tab appears.

4. Click Test Environments to define the test environments to be added to the library.

5. Click Create. The Test Environment Properties page appears.

6. Enter a name and description for the test environment.

7. Click the Application drop-down list and select the application to be tested from the list.

8. Enter any tags to use as keywords to further identify this test environment. These tags make test environments easier to identify and to understand the purpose of the environment.

9. Click the magnifying glass icon and select the Deployment Plan to use for this test environment. The Deployment plan must be created on a zone to which the Test Designer has privileges on Oracle Virtual Machine and Operator. For information regarding deployment plans, see Oracle Enterprise Manager Lifecycle Management Administrator's Guide.

10. Click the Select web-application drop-down and select the appliance that will act as the application or web server when the environment is deployed in the cloud.

11. Click Save. The test environment is added to the library. Click on a test environment name to drill down to the Test Environment Details page. Click on a deployment plan name to drill down to the Deployment Plan Details page.

Verifying Available Test Assets and Environments

The Testing Cloud Self Service Portal allows Testers to verify which test assets and test environments have been assigned to them by the Testing Administrator.

Note: Ensure the prerequisites are met before verifying available test assets and environments (see Prerequisites).

Follow this process to verify test assets and test environments assigned to your role.

1. Log in to Enterprise Manager Cloud Control using Tester credentials. You are logged into Self Service Portal.

2. Click My Tests. The Testing Cloud Self Service Portal Home tab appears.

3. Click the My Library tab. The My Library tab shows the test environments and test assets published to you by the Testing Administrator.

4. Click Test Environments.

5. Click on an environment to view the details about the environment. Click on the component to drill down to the Assembly Structure details.

6. Click Test Assets.

7. Click on a test asset to view the details about the asset.

Creating Tests

TaaS allows Testers to create tests from test assets and environments to perform testing activities. Testers create tests using test assets and environments that have been assigned to them by the Testing Administrator.

Note: Ensure the prerequisites are met before creating Tests (see Prerequisites).

Follow this process to create Tests.

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

2. Click My Tests in the Self Service Portal.

3. Click the My Tests tab in the Testing Cloud Self Service Portal.

4. From the Create menu, select the type of test you want to create, either Oracle Load Testing or Custom. The Create Test page appears.

5. For the Custom test, select the Test Driver type.

   Enter a name and description for the test.

6. Click the magnifying glass icon, select the test asset to be used in this test, and click Select.

7. Select the test asset, review the details, and click Continue.

8. Select the Source and click the magnifying glass icon and select the test environment to be used in this test.

9. Select the test environment and click Select. The Mapping URLs settings control shows all the URLs found in the load-testing scenario contained in the asset.

10. From the options column, select From Appliance. You will see an appliance (a virtual machine) that will be used to replace the "source URL" that was in the load testing scenario.

11. Click the Test Driver drop-down and select the test driver to use to run the test in the cloud.

12. Click OK. The test is added to the library that specifies an asset, an environment, and a test-driver. Click on a test name to drill down to the details page.

Creating and Running Trials

The Testing Cloud Self Service Portal allows Testers to create and run trials from the test instances that have been created in the portal. The Tester runs trials on the test asset and environment, monitors ongoing trials, and analyzes the results of completed trials.

Note: Ensure the prerequisites are met before creating and running trials (see Prerequisites).

Follow this process to create and run trials.

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

2. Click My Tests in the Self Service Portal.

3. Click the My Tests tab in the Testing Cloud Self Service Portal.

4. Select a test to run for this trial.

5. Click Create in the Trials section. The Create Trial page appears.

6. Enter a name and description for the trial.

7. Specify the Application Under Test instance details.

   • Creating new instance: Users can create new virtual machines on the cloud for the application under test.

   • Re-using an existing instance: If users have created instances for this test or this application before, they can choose to re-use the same set of virtual machines instead of creating new ones.

     Re-using an existing instance is faster.

8. Enter a password of your choice for the Root Password and reenter to confirm it if creating a new instance.

   Important:

   This value is not retrievable. DO NOT FORGET THIS PASSWORD.

9. Specify the Stop After and Delete After settings and click Continue.

10. Enter the number of virtual users to run for this trial in the Trial Virtual Users column of each script.

    You can set these values ONLY if the Test Designer (who created the asset) has chosen to allow users to override the values. Otherwise, the application uses the information available in the asset and these controls are grayed out.

11. Specify the Ramp Up Rate settings for this trial.

    The Ramp Up Rate is the rate by which the load increases during the load test. You can set these values ONLY if the Test Designer (who created the asset) has chosen to allow users to override the values. Otherwise, the application uses the information available in the asset and these controls are grayed out.

12. Set the Test End Conditions settings for the trial and click Continue.

    You can set these values ONLY if the Test Designer (who created the asset) has chosen to allow users to override the values. Otherwise, the application uses the information available in the asset and these controls are grayed out.

    If you do not set the test end conditions, the test runs until it is stopped manually.

13. Select the test asset, review the details, and click Continue.

14. Enter an instance name for the test driver.

    You can create a new virtual machine for the test driver or re-use an existing instance. If the existing instance is being used by another trial, it will not be available.

15. Enter a password of your choice for the Master Password and reenter to confirm it. This is the password Testing Cloud Self Service Portal will set for the Oracle Load testing Virtual Machine and all accounts for Oracle Load Testing.

16. Specify the Stop After and Delete After settings and click Continue.

17. Specify the Schedule for starting the trial run and click OK. A new trial will be created and submitted for execution. The initial execution of a trial will take some time to deploy the Virtual Machines, copy necessary software, and start the test.

Monitoring On-Going Trials

The Testing Cloud Self Service Portal allows Testers to monitor running trials and create charts of the key performance indicators for the application-under-test.

Note: Ensure the prerequisites are met before monitoring on-going trials (see Prerequisites).

Follow this process to monitor on-going tests.

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

2. Click My Tests in the Self Service Portal.

3. Click the My Tests tab in the Testing Cloud Self Service Portal.

4. Select the test to view the trial details. The trial status information appears in the Trials pane.

5. Click on the trial name link in the Trials pane to navigate to the trial-home page.

   You can monitor the progress of the trial by clicking Status. If the status of the trial is In Progress, click the trial name which will take you to the trial home and the remaining trial information.

6. Expand the metric nodes and select metrics or counters to plot them in the chart.

Comparing Trial Results

The Testing Cloud Self Service Portal allows Testers to compare trial results and create charts of the key performance indicators for the application-under-test.

Note: Ensure the prerequisites are met before comparing trial results (see Prerequisites).

Follow this process to compare trial results.

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

2. Click My Tests in the Self Service Portal.

3. Click the My Tests tab in the Testing Cloud Self Service Portal.

4. Select the test to view the trial details. The trial status information appears in the Trials pane.

5. Select two trials of the same test that have been previously run.

   To select 2 trials of the same test, select one trial, then click Shift/Control to select the other trial. The trials must belong to the same test.

6. Click Compare Results. You can compare the metrics of the two trials to view the differences.

   When in this mode, selecting any metric will chart two lines - one for each of the selected trials. You can also compare the Oracle Load Testing session summary report side-by-side.

Share Tests

To avoid duplication of effort, you can share tests with other users. Note that the person who created and shared the tests will be charged for any resource usage associated with the shared tests.

To share tests, perform the following steps:

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

2. Click My Tests in the Self Service Portal.

3. Click the My Tests tab.

4. Select a test from the list of Tests on the My Tests page.

5. Click Share.

6. Select a user to share the Test with.

7. Click Share.

8. Click Log Out.

To verify that the test is shared:

1. Log in as the user whom you shared the test with.

2. Verify that you have access to the shared test.

Viewing Activity and Usage Information

The Testing Cloud Self Service Portal allows Testers to view summary information for test assets and test environments available to you, your quota as defined by the Testing Administrator for your role, your current utilization of hardware resources, and submitted and scheduled trials.

Note: Ensure the prerequisites are met before viewing activity and usage information (see Prerequisites).

Follow this process to view activity and usage information.

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

2. Click My Tests in the Self Service Portal.

3. Click the Home tab in the Testing Cloud Self Service Portal. This tab shows summary information for test assets and test environments available to you, your quota as defined by the Testing Administrator for your role, your current utilization of hardware resources, and submitted and scheduled trials.

Viewing Deployment Information

The Testing Cloud Self Service Portal allows Testers to create tests from test assets and test environments to perform testing activities using the Testing Cloud Self Service Portal. Testers create tests using test asset and environment that have been assigned to them by the Testing Administrator. The Tester runs trials on the test asset and environment, monitors ongoing trials, and analyzes the results of completed trials.

Note: Ensure the prerequisites are met before viewing deployment information (see Prerequisites).

Follow this process to view activity and usage information.

1. Log in to Enterprise Manager Cloud Control using Testing Cloud Self Service Portal Tester credentials. You are logged into Self Service Portal.

2. Click My Tests in the Self Service Portal.

3. Click the My Deployments tab in the Testing Cloud Self Service Portal. This tab shows a table of all environments and test-drivers deployed for you.

4. Select a test environment or test driver instance. The details pane shows the appliances within the selected test environment or test driver instance.

5. Click on an appliance within the selected test environment or test driver instance to navigate to the details view of the appliance.

Browsing the Testing Home Page

The users HOME tab displays a summary of what the user has done, including resources consumed for testing and tests created by the user. From the HOME tab, users can quickly access previously executed test trials in order to compare results or to do more in depth post-run analysis.

The Testing home page provides information regarding the statistics of ongoing testing. General statistics are provided, as well as test assets and environments, ongoing test and trials, Oracle Load Testing, and the Infrastructure. It also provides an overview of the servers and infrastructure resources consumed by the user, like the number of servers provisioned, number of CPUs, as well as RAM and Storage.

When you log in as Test Designer, you only see the My Library tab. When you log in as Test Administrator, you see the Home tab. When you log in as a Super Administrator, you see both the Home and My Library tabs.

The sections on the Home tab are:

• General - Total counts of various artifacts available in the system, for example, its “total tests present in system”, “total trials ran across all trials”, and so on.

• Test Assets/Environments - Accumulation graph of test assets, environments, and machines in use in the Testing Cloud. Count and trend data are included. This is the Test Administrator's overview of the entire TaaS portal, regardless of Testers.

• Tests and Trials - Daily totals of the number of tests created or daily totals of the number of trial runs.

• Infrastructure - Daily consumption of infrastructure resources: CPU, Memory, and Disk space storage.

• Oracle Load Testing (OLT) - Daily total of the number of OLT Virtual Users (VU) that were run. Oracle Load Testing uses OpenScript scripts to simulate users accessing an application under test.

The vertical axis shows daily usage totals based upon the View selection in each of the regions.

The My Library tab provides details on both Test Assets and Test Environments.

1. Log in to Enterprise manager using Tester credentials.

2. Click My Tests radio button.

3. Click the Home tab.

You will see a summary of test assets and test environments available to you, your quota as defined by the Test Administrator for your role, your current utilization of hardware resources, and the submitted and scheduled trials.